Deep Mutual Information Maximin for Cross-Modal Clustering

Cross-modal clustering (CMC) aims to enhance the performance by exploring complementary information from multiple modalities. However, performances of existing CMC algorithms are still unsatisfactory due conflict heterogeneous modalities and high-dimensional non-linear property individual modality. In this paper, a novel deep mutual maximin (DMIM) method for cross-modal is proposed maximally preserve shared while eliminating superfluous in an end-to-end manner. Specifically, multi-modal encoder firstly built align latent feature distributions sharing parameters across Then, DMIM formulates complementarity multi-modalities representations as objective function, which identified maximization minimization respectively. To solve we propose variational optimization ensure it converge local optimal solution. Moreover, auxiliary overclustering mechanism employed optimize structure introducing more detailed classes. Extensive experimental results demonstrate superiority over state-of-the-art methods on IAPR-TC12, ESP-Game, MIRFlickr NUS-Wide datasets.